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We report on the study of the fractional quantum Hall effect at the filling factor 5/2 using 
exact diagonalization method with torus geometry. The particle-hole symmetry breaking effect is 
considered using an additional three-body interaction. Both Pfaffian and anti-Pfaffian states can be 
the ground state depending on the sign of the three-body interaction. The results of the low-energy 
spectrum, the wave function overlap, and the particle-hole parity evolution, have shown the clear 
evidence of a direct sharp transition (possibly first-order) from the Pfaffian to the anti-Pfaffian state 
at the Coulomb point. A quantum phase diagram is established, where one finds further transitions 
from the Pfaffian or anti-Pfaffian state to the nearby compressible phases induced by a change of 
the pseudopotential. 

PACS numbers: 73.43.-f, 73.22.Gk, 71.10.Pm 
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The fractional quantum Hall effect (FQHE) at the fill- 
ing factor v — 5/2 has recently drawn intensive attentions 
in theoretical and experimental studies [l|, |2], |3j, U, Ifa, |g, l3, 
SHE EH, El- The Moore-Read Pfaffian (Pf) state has 
been proposed as a successful candidate to describe the 
5/2 FQHE [1, [1, II] . The Pf state as a non-Abelian topo- 
logical phase suggests a potential application towards the 
quantum computing However, the Pf state breaks 
particle-hole (PH) symmetry and its PH conjugate state, 
known as the anti-Pfaffian (APf) state, has been sug- 
gested to be another candidate state 11 1 . In the limit 
of vanishing Landau-level (LL) mixing, the Pf and APf 
model wave functions have the same energy for the 5/2 
FQHE system with the pure Coulomb interaction as the 
system has the PH symmetry. Numerically the ground 
state (GS) of such a system appears as a superposition 
of the Pf and APf states @, However, the PH sym- 
metry of the Hamiltonian in real systems can be broken 
by many factors, such as LL mixing. This raises a new 
challenge regarding the nature of the GS realized in the 
experimental system (lfl 11 1, which can be the Pf, APf, 
or superposition of them. To address this issue, we add 
a PH nonsymmetric three-body (3b) interaction together 
with the Coulomb interaction as the model Hamiltonian 
to reexamine the nature of the GS and to study the quan- 
tum phase transitions (QPT) between distinct quantum 
phases. 

In this letter, We investigate the low-energy states of 
the 5/2 FQHE system using Lanczos method for finite- 
size systems with up to N e = 14 electrons. The GS of a 
Coulomb system is found to have a transition into the Pf 
or APf state with the turn-on of the 3b interaction. The 
Pf and APf states are robust in a range of the 3b interac- 
tion, insensitive to the detailed form of the 3b interaction 
as long as the PH symmetry is broken. The calculations 
on the GS wave function overlap, energy and PH parity 
evolutions show the strong evidence that the phase tran- 
sition between the Pf and the APf state appears to be 
first-order, occurring exactly at the Coulomb point. Un- 



der an extra short-range pseudopotentialQ, Pf and APf 
states can have transition to the stripe phase or compos- 
ite fermion liquid (CFL) phase. 

We consider a two-dimensional electron system under 
a perpendicular magnetic field. Periodic boundary con- 
ditions for magnetic translational operators are imposed 
with a quantized flux TV^ through a rectangular unit cell 
x L 2 . The magnetic length I is taken as the unit 
of the length and the energy is in units of e 2 /4-7re£ To 
reduce the size of the Hilbert space, we carry out our cal- 
culation at every pseudomomentum K = (K\,K2) [13l |. 
where Ki{K2) is in unit of 2ir/ Li(2w/ L2) and the even 
number of electrons is used for N e . The magnetic field is 
assumed to be strong enough so that the spin degener- 
acy of the LLs is lifted[3j, |4|, |8|. One can thus project the 
system Hamiltonian into the topmost, half filled, N = 1 
LL The projected Hamiltonian for the Coulomb in- 
teraction has the form: 
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where V m is the Haldane's pseudopotential of the 
Coulomb interaction at N = 1 LL and L m (x) is the La- 
guerre polynomial. The momentum q is taken discrete 
values suitable for the unit cell lattice, is the guiding 
center coordinate of the i-th electron. 

The Pf state on torus can be obtained as the zero- 
energy GS of a repulsive 3b potential given by [3] : 



Ho = - 



i<j<k 
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where Si is a symmetrizer. Besides the center-of-mass 
degeneracy, the Pf state appears at three particular pseu- 
domomenta of (0, N e /2), (N e /2, 0), (N e /2, N e /2) [lj as 
degenerate ground states. To study the effect of the PH 
symmetry breaking, we set up a model Hamiltonian with 
the form H = H c + V3& • H 3 , where the parameter V31, can 
change its sign and magnitude. 
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FIG. 1: (Color online) Low-lying energies as a function of 
the pseudomomentum \J Kf + K\ (in physical units) for a 
N e = 12 pure Coulomb system with the rectangular unit 
cell at the aspect ratios of (a) L1/L2 = 0.99, and (b) 
L1/L2 — 0.97, where a pair of lowest-energy states at the 
pseudomomentum (0,6) are nearly degenerate. The plots of 
(c) and (d) are low-lying energies (relative to the globe GS) 
as a function of Vsb for the N e = 12 model system with the 
additional 3b interaction H3 and H£, respectively. The unit 
cell is the same as the one in (b). The circle, square and tri- 
angle symbols stand for the GSs (solid symbols) and the first 
excited states (open symbols) at the pseudomomenta of (0,6), 
(6,0) and (6,6), respectively. The insets in (c) and (d) are the 
zoom-in spectra for —0.1 < V36 < 0.1. 



Wc first present the low-energy spectrum as shown in 
Fig. 1(a) and Fig.l (b) for a pure Coulomb system with 
N e = 12 electrons. The low-energy sector consists of 
six states from the three pseudomomenta of (0,6), (6,0) 
and (6,6), three local GSs of which are termed as the 
GS triplet, separating from the other excited states. The 
wave function overlaps with Pf and APf states demon- 
strate that these six states are closely related to the su- 
perposition states of Pf and APf. By adjusting the as- 
pect ratio of the unit cell from L1/L2 = 0.99 to 0.97 the 
lowest-energy pair states at the pseudomomentum (0,6) 
change from well separated to nearly degenerate (see also 
the inset of Fig. 1(c) and (d)). This is consistent with the 
fact that the tunneling between Pf and APf in a finite- 
size Coulomb system is sensitive to the specific geometry 
of the system. The fine tuning of the aspect ratio reduces 



the finite-size effect by reducing the energy gap between 
the lowest two states at the particular pseudomomentum. 

Fig. 1(c) exhibits the low- lying excitation spectrum 
as a function of for the N e = 12 model system with 
the additional 3b interaction H3. The unit cell is the 
same as the one described in Fig. 1(b). At the Coulomb 
point {Vsb = 0), each GS in the GS triplet and its first 
excited state have the opposite PH parity. In the region 
of Vsb > 0, with H 3 strength getting stronger, the de- 
generacy of the GS triplet improves and the energy gap 
between the GS triplet and other states increases. In the 
region of V3& < 0, with the 3b strength |V3fc| < 0.1 the 
near degeneracy of the GS triplet maintains. With more 
negative V36, the energy width of the lowest triplet in- 
creases and these energy levels cross with higher energy 
states, indicating a phase transition induced by —H3. 

We further check if the obtained results depend on 
the precise form of the 3b interaction. Noting that H3 
has the PH symmetric and anti-symmetric components, 
we extrapolate the PH anti-symmetric component of H3, 
which has the form of H3 = (H3 — i?3)/2 with H3 as the 
PH conjugate of H 3 , as the other type of additional 3b 
interactions. This way, H = H c + V3& ■ H£ has the generic 
form of both symmetric and anti-symmetric parts. The 
obtained low-energy spectrum is shown in Fig. 1(d). Due 
to the PH anti-symmetry of the H3 , the spectrum is sym- 
metric around the Coulomb point. Clearly, the three 
lowest energy states are well separated from the excited 
states, indicating the establishing of Pf or APf states. 

To explore the nature of the GS under the additional 
3b interaction, in Fig. 2(a) we plot the wave function 
overlap (squared) between the GS triplet and a Pf or an 
APf state as a function of V 3 b for the N e = 12 model 
system with the 3b interaction H3 and the same unit 
cell as in Fig. 1(b). In the positive V$h region, with the 
H 3 strength growing, the GS projection on the Pf state 
monotonously increases towards unitary while the pro- 
jection on the APf state decreases and drops to a small 
value, indicating that the GS is in the same class as the 
Pf state. In the negative region, with the strength of 
— H3 increasing, the projection on the Pf state quickly 
decreases and tends to be zero. On the other hand, 
the GS projection on the APf state, up to the strength 
IV3&I ~ 0.1, increases and remains at a finite value be- 
tween 0.6 and 0.7, indicating that the GS is associated 
with the APf in this region. When the strength increases 
further, the GS projection on the APf state continuously 
drops and eventually vanishes. The results for the model 
system with H3 and the same unit cell have been shown 
in Fig. 2(b). In the positive region, the projections 
on the APf state tend to vanish as the 3b strength grows 
while the projections on the Pf state increase to some 
finite value between 0.6 and 0.7 with the strength up to 
0.1, indicating the GS is associated with the Pf state. In 
the negative V3b region, the GS has large overlap with 
the APf phase. The above characteristics from the wave 



3 



— •— Pf-c 


-APf(0,6) 




(a) 


2^ M 


— -HPf— □— APf(6,0) 




i — *-Pf— a- APf(6,6) 






ED — O— 0- 



• — Pf— °- APf(0,6) 



(b) 




1=1=1=1=1=, 



=■=•=( 

— Pf— □— APf(6,0) 

Pf— a— APf(6,6) 




-♦— Pf: Ne=10 
— O— APf:Ne=10 
— ■— Pf: Ne=12 
-□-APf:Ne=12 
— a— Pf: Ne=14 
— a— APf:Ne=14 



1 



FIG. 2: (Color online) The projection of the system GS on 
the Pf and APf states as a function of V-si,. The aspect ratio 
Li/L 2 are 0.94, 0.97 and 0.70 for the system with N e =10, 12 
and 14 electrons, respectively. The curves with solid (open) 
symbols stand for the projection on the Pf (APf) state. States 
of N e = 12 system with the additional 3b interaction of (a) 
H3 and (b) H3 at pseudomomenta (0,6), (6,0) and (6,6) are 
represented by the curves with circle, square and triangle sym- 
bols, respectively, (c) States of N e =10 system with H3 at 
pseudomomentum (5,5), N e = 12 at (0,6) and N e = 14 at 
(7,7) are represented by the curves with diamond, square and 
triangle symbols, respectively. 



function overlaps agree with the spectrum feature shown 
in Fig. 1(c) and (d). We also notice that the projection 
curves of the local GS at the pseudomomentum (0,6), 
which has the nearly degenerate first excited state, ex- 
hibit the sharpest transition in a small region of V36 cross- 
ing the Coulomb point, indicating the strongest QPT sig- 
nal. In the following discussion we target such particular 
GS from the GS triplet to investigate the phase transi- 
tion between the Pf and the APf state. In Fig. 2(c), the 
results of the wave function overlap for different sizes of 
the system with the additional 3b interaction H3 have 
been demonstrates. For all systems considered, the Pf 
state dominates on the V%b > side while the APf states 
dominates on the V36 < side as long as the 3b interac- 
tion strength is smaller than 0.1 and a sharp transition 
occurs at the Coulomb point. 

The low-lying energy spectrum and the GS wave func- 
tion projections provide us with a convincing picture for 
the existence of a QPT towards two different states, the 
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FIG. 3: (Color online) Detectors of the QPT for the model 
system with H3 at —0.1 < Vsb < 0.1 and different iV e . (a) 
GS fidelity function, (b) second derivative of the GS energy, 
and (c) GS PH parity as function of V31, ■ The curves with 
diamond, star and triangle symbols represent for the system 
with N e —10, 12 and 14 electrons, respectively. 



Pf and APf states, with the turn-on of the additional 3b 
interaction. To gain the further understanding for this 
QPT, we study several physical quantities for a model 
system near the Coulomb point. In Fig. 3 we show 



the evolutions of the fidelity function [14j, energy and 
PH parity of the chosen GS for different system sizes 
when the strength of the 3b interaction H 3 varies in the 
range |V3b| < 0.1 with the increment SV — 0.01. The 
pseudomomenta of the chosen GSs and the unit cells for 
N e = 10, 12, 14 systems are the same as those in Fig. 
2(c). The GS fidelity function, which we use in the plot 
of Fig. 3(a) to probe the response of the wave func- 
tion to the variation of the parameter V^b, has the form 
of F(V 3b ) = - MMVp>+gfflQ , The GS wave 
function is found insensitive to the change of the 3b in- 
teraction when its strength is larger than 0.02. However, 
within the strength range | V3& | < 0.02, the value of the 
fidelity function abruptly increases, indicating a QPT oc- 
curs. The peak around V^b = becomes sharper if we 
improve the degeneracy between the chosen GS and its 
first excited state by carefully tuning the parameters of 
the unit cell. With the sharp peak located at V31, = for 
all different system sizes, we can identify the Coulomb 
point as the transition critical point. We can also trace 
the GS energy to probe a QPT. In Fig. 3(b) we plot the 
second derivative of the GS energy as a function of V36. 
For all the system sizes considered, the curves exhibit the 
singularity- like behavior at the Coulomb point, signaliz- 
ing a first-order QPT. Another useful protocol for us to 
understand the QPT crossing V^b — is the PH parity 
of the GS as shown in Fig. 3(c). Within a narrow range 
of IV3&I < 0.03, the PH parity of the GS quickly collapses 
from the Coulomb point, where it is either unitary or 
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FIG. 4: (Color online) Phase diagram of iV e = 12 system 
near the Coulomb point with the 3b interactions of (a) H3 
and (b) H3 . The letters of S, L, Pf and APf are used to mark 
the phase regions of (SVijVkb) with the stripe-like, composite- 
fermion-liquid-like, Pf and APf phase, respectively. 



with a positive (negative) sign, the boundary point 6Vf 
locates where the GS projection on the CFL state be- 
gins to exceed the one on the Pf (APf) state with the 
square unit cell considered. As shown in the Fig. 4(a), 
on the 5Vi > side both the Pf and the APf phase tend 
to expand their phase boundaries with the CFL phase 
when the 3b interaction strength grows. On the SV\ < 
side, the Pf phase extends its boundary with the stripe 
phase while the APf phase shrinks. The nonsymmetric 
behaviors of the Pf and APf phase result from the PH 
symmetric component of H% . The phase diagram of the 
system with 3b interaction H^, as shown in Fig. 4(b), is 
symmetric around the phase line = 0. 

In summary, the PH nonsymmetric 3b term modeling 
a realistic 5/2 FQHE system can bring either the Pf or 
the APf state as the ground state depending on its sign. 
The pure Coulomb system is at the critical point for a 
possible first-order transition between these two states as 
one changes the sign of the 3b interaction. Our results 
sug gest that the APf state is indeed a valid candidate fiol. 
llj for the experimental observed 5/2 FQHE. 



ACKNOWLEDGMENTS 



anti-unitary, confirming the relation between the QPT 
and the PH symmetry breaking. 

In a generic system with the finite LL mixing, one 
would expect some effective interaction terms breaking 
the PH symmetry, which are now modeled by H£. One 
would also expect some effective terms with the PH sym- 
metry, which may be modeled as a change of the Hal- 
dane's pseudopotential We map out the quantum 
phase diagram for the 5/2 FQHE system in the presence 
of the 3b interaction and an extra short-range pseudopo- 
tential SV\. Fig. 4(a) and (b) show the phase diagram 
of the N e — 12 model system with the 3b interaction H3 
and i?3, respectively. We only concern the 3b interac- 
tion with the strength less than 0.1, where the obtained 
phase is most relevant for a Coulomb system with the 
weak PH symmetry breaking. Similar to the story in a 
pure Coulomb system we find that the model system 
with the weak additional 3b interaction will transfer from 
the Pf-like or APf-like state to the compressible stripe 
phase when we reduce the pseudopotential SV\ and to 
the CFL phase when we increase 8V\. Between these 
two compressible phases, the Pf phase is found associ- 
ated with the positive and the APf phase with the 
negative V^. For the boundary between the stripe and 
the Pf/APf phase, at each the boundary point SVf 
locates where the peak value of the structure factor as a 
function of 8V\ drops most dramatically. For the bound- 
ary between the Pf (APf) and CFL phases, at each V36 
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